Skip to content

Fix multi FlatMap scale and size#2669

Merged
wasphin merged 1 commit into
apache:masterfrom
chenBright:fix_multi_flat_map
Jul 11, 2024
Merged

Fix multi FlatMap scale and size#2669
wasphin merged 1 commit into
apache:masterfrom
chenBright:fix_multi_flat_map

Conversation

@chenBright

@chenBright chenBright commented Jun 21, 2024

Copy link
Copy Markdown
Contributor

What problem does this PR solve?

Issue Number:

Problem Summary:

  1. Multi模式下,插入数据发生hash冲突的时候,size没有加一。另一方面,插入数据不能自动扩容。
  2. m1 = m2,m1扩容后m1._nbucket不等于m2._nbucket的时候,m1的_buckets上除了插入数据的位置,其他位置没有set_invalid,后续使用会出现内存问题。
  3. 扩容时,_buckets和_thumbnail都是先释放再申请新的内存。如果申请失败了,FlatMap就完全不可用了。

What is changed and the side effects?

Changed:

  1. size++。另外,当FlatMap负载超过阈值,则遍历Bucket链表,当存在不同的key的时候,则扩容。否则,不扩容直接插入。因为key一样,扩容后这些数据还是在同一个链表上。
  2. 扩容后都要初始化_buckets和_thumbnail数组。
  3. 申请内存成功后,再将其复制给_buckets和_thumbnail。一旦出现错误,则先释放已经申请的内存再返回失败。

Side effects:

  • Performance effects(性能影响):

  • Breaking backward compatibility(向后兼容性):


Check List:

  • Please make sure your changes are compilable(请确保你的更改可以通过编译).
  • When providing us with a new feature, it is best to add related tests(如果你向我们增加一个新的功能, 请添加相关测试).
  • Please follow Contributor Covenant Code of Conduct.(请遵循贡献者准则).

@chenBright chenBright force-pushed the fix_multi_flat_map branch 2 times, most recently from 2ddda80 to 6819bb7 Compare June 23, 2024 15:20
@chenBright chenBright force-pushed the fix_multi_flat_map branch from 6819bb7 to 0f4a237 Compare June 23, 2024 17:43
@wwbmmm

wwbmmm commented Jul 8, 2024

Copy link
Copy Markdown
Contributor

LGTM

@wasphin

wasphin commented Jul 10, 2024

Copy link
Copy Markdown
Member

这个是不是可以合了?

@chenBright

Copy link
Copy Markdown
Contributor Author

这个是不是可以合了?

是的,可以合了。

@wasphin wasphin merged commit 8b5571f into apache:master Jul 11, 2024
@chenBright chenBright deleted the fix_multi_flat_map branch July 11, 2024 10:25
@zhaojinzhou

zhaojinzhou commented Aug 24, 2025

Copy link
Copy Markdown

这个 MR有问题吧。
非常低级 的错误。

image

1be060d

在这个MR中修改了 find_power2
导致 flatmap在赋值的时候 非常 常见的 死锁。 只要bucket小于8或者 2的次幂 都会死锁。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants